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PROJECT  DESCRIPTION 


Implicit  systems  of  differential  equations  of  the  form  F(t,x,x')  =  0  naturally 
arise  in  many  circuit  and  control  problems,  economic  models,  and  the  solution  of 
partial  differential  equations  by  the  method  of  lines.  Implicit  systems  are  also 
called  singular,  differential -algebraic,  semi-state,  constrained,  and  descriptor. 
The  theory  is  well  understood,  and  numerical  codes  exist,  for  index  zero,  index 
one,  and  linear  constant  coefficient  problems.  Higher  index  implicit  systems  occur 
in  circuit  and  control  problems.  The  numerical  and  analytic  behavior  of  such 
higher  index  systems  is  not  well  understood  and  is  incomplete.  It  has  recently 
been  shown  that  traditional  methods,  such  as  backward  differentiation,  need  not 
work  on  higher  index  systems.  Good  characterizations  of  the  solution  manifold  are 
often  difficult  to  obtain.  This  research  project  is  to  study  the  numerical  and 
analytic  solution  of  higher  index  implicit  differential  equations.  Applications 
will  be  made  to  circuit  theory,  control  theory,  and  the  analysis  of  numerically 
ill-conditioned  index  one  systems.  - 
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During  the  report  period  the  following  papers  were  written  as  part  of  this 
research  effort: 

PI.  S.L.  Campbell,  Rank  deficient  least  squares  and  the  numerical 
solution  of  linear  singular  implicit  systems  of  differential 
equations,  in  Linear..  Algebra  and  Its  ..Role  in_Svstems  .Theory, 

AMS  Cantempory  Mathematics  Series,  Vol  47(1984),  51-63.  (Invited 
paper) 

P2.  S.L.  Campbell,  Index  two  linear  time  varying  singular  systems  of 

differential  equations  ll,  Circuits. Systems,  and  Signal  Processing. 
5(1986),  97-107. 

P3.  S.L.  Campbell,  Ronlmear  time-varying  generalized  state-space  systems : 
an  overview,  Ezqq  L__23_1FEF.  Conference  on  Decision  and  Control.  (1984), 
268-276  (Invited  paper) 

P4.  S.L.  Campbell,  Consistent  initial  conditions  for  linear  time  varying 
singular  systems,  in  Frequency  Domain  and  State  Space  Methods  for 
Linear  Systems,  edited  by  C.I.  Byrnes  and  A.  Lindquist,  Elsevier, 

1986,  313-318. 


P5.  K.D.  Clark,  Difference  methods  for  the  numerical  solutions  of 

time-varying  singular  systems  of  differential  equations,  SIAM  J. 
Algebraic  &  Discrete  Methods.  7(1986),  236-246. 

P6.  S.L.  Campbell,  A  general  form  for  solvable  linear  time  varying 
singular  systems  of  differential  equations,  SIAM  J.  Mathpmatiral 
Analysis .  (to  appear) 

P7 .  K.D.  Clark,  The  numerical  solution  of  some  higher  index  time 

varying  semi-state  systems,  Circuits  Systems  and  Signal  Processing. 

(to  appear) 

P8 .  K.D.  Clark,  The  numerical  solution  of  linear  time  varying  singular 

systems  of  differential  equations  by  difference  methods,  Ph.D.  Thesis. 
North  Carolina  State  University,  1986. 

P9.  S.L.  Campbell,  The  numerical  solution  of  higher  index  linear  time 

varying  singular  systems  of  differential  equations  II,  (preprint)  1986. 

P10.  S.L.  Campbell,  Operator  methods  and  singular  control  problems  in 

Operator  Methods  for  Control  Problems.  Marcel  Dekker,  1986,  (to  appear) 
(invited  paper) 

Pll.  S.L.  Campbell ,  Bilinear  nonlinear  descriptor  control  svstems,  Proc . 

SIAM  Conferences  cn  Systems  and  Signals  ‘(to  appear)  1986. 

P12.  K.D.  Clark,  A  structural  form  for  higher  in>Jex  semistate  equations  r : 
theorv  and  applications  to  circuit  and  control  theory  (preprint)  1986. 

Additional  references  of  the  principal  investigator  [Ci]  aixi  others  (Ri]  appear 

at  the  end  of  this  report. 


If  a  system  of  differential  equations  is  written  in  the  semi-explicit  form  as 


u1  =  f(u,v,t) 

(la) 

0  =  g(u,v,t) 

(lb) 

then  the  system  (1)  is  called  index  one  if  is  nonsingular.  In  this  case  the 

O  V 

algebraic  constraint  (lb)  may  be  solved  for  v,  to  give  v  =  *(u,t)  and  (la)  becomes 
an  explicit  equation  u*  *  f(u,  *(u,t),  t) .  However,  in  some  control  applications 
[Cl],  [C7],  [Rl],  [R2],  [R7],  [Rll],  [R12]  the  matrix  is  always  singular.  These 
problems  are  called  higher  index  problems.  Definitions  of  index  for  the  more 
general  system 


F(X,x' , t)  =  0  (2) 

appear  in  [C5].  Traditionally  (1)  has  been  numerically  solved  using  backward 
differentiation  formulas  (BDF)  [R3],  [R4].  Recently,  it  has  been  shown  that  BDF's 
need  not  converge  for  either  nonlinear  [C5]  or  linear  time  varying  [R4], 


higher  Index  systems.  Prior  to  the  work  of  the  principal  investigator,  no  general 
method  for  the  numerical  solution  of  (3)  existed. 

An  approach  to  numerically  solve  (3)  was  proposed  in  [C4] .  However,  numerous 
questions  on  the  practical  implementation  and  theoretical  implications  of  this 
method  remained  unresolved.  The  approach  was  shown  in  [C4]  to  be  theoretically 
applicable  in  several  cases  of  interest,  but  the  range  of  applicability  was 
uncertain . 

The  key  idea  in  [C4]  was  as  follows.  Suppose  that  (3)  is  solvable  on  an 
interval  I.  That  is,  for  every  sufficiently  smooth  input  f  there  is  at  least  one 
solution  x  and  the  solutions  for  a  given  f,  are  uniquely  determined  by  their  value 
at  any  time  tel.  From  [C2]  it  is  known  that  the  solution  can  depend  not  only  an 
A,  B,  f  but  also  their  derivatives.  Thus  some  differentiation  is  necessary. 

Assume  that  A,B,f  are  sufficiently  smooth  on  l  and  at  time  tel,  let  (A^) ,  (B^, 
(xi),  {fi>  be  the  Taylor  coefficients  of  A,  B,  x,  and  f  respectively.  These 
functions  do  not  have  to  be  infinitely  differentiable,  just  sufficiently 
differentiable.  Then  for  any  j  >  0,  any  tel,  equation  (3)  implies  that  the 
coefficients  satisfy 


or  more  compactly 


In  [C4]  it  is  shown  that  for  all  then  known  classes  of  solvable  systems  that  if  j 
is  appropriately  chosen,  then  (5)  uniquely  determines  x^  even  though  x^  is  not 
unique.  Since  x1  =  x' (t)  it  is  new  possible  to  define  a  method  to  integrate  (3) . 

This  general  approach  was  refined  and  improved  by  the  principal  investigator 
in  [PI].  This  paper  made  several  contributions.  First,  it  examined  the  numerical 
considerations  in  solving  (5)  and  showed  hew  to  do  so  in  a  numerically  reliable 
manner.  Secondly  it  greatly  reduced  the  computational  effort  of  the  method 
proposed  in  [C4]  by  showing  how  (5)  could  be  used  to  define  an  explicit 
differential  equation  x‘  =  R(t)x  +  r(t)  at  each  time  step.  This  made  possible  the 
definition  of  both  Adams  and  Runge-Kutta  methods.  Due  to  the  smaller  sized 
coefficient  matrices  involved,  these  methods  were  more  computationally  efficient 
than  the  higher  order  Taylor  methods  of  [C4]  by  an  order  of  magnitude. 

One  previous  difficulty  with  many  of  the  approaches  for  solving  (3),  including 
the  BDF,  has  been  the  lack  of  general  methods  for  finding  the  manifold  of 
consistent  initial  conditions.  In  [P4]  the  principal  investigator  showed  that  for 
the  classes  of  systems  covered  by  [PI],  that  the  equations  (5)  could  be  used  to 
develop  an  algorithm  for  calculating  the  consistent  initial  conditions.  This  is 
the  first  general  method  for  doing  so  based  strictly  on  the  coefficients  of  A,B,  f 
and  not  requiring  a  series  of  time-varying  coordinate  changes. 

Although  the  results  reported  so  far  made  substantial  progress  on  the  analysis 
of  the  linear  time  varying  system  (3),  severed  major  difficulties  remained.  Among 
these  were  the  lack  of  either  any  explicit  criteria  for  solvability  or  good 
descriptions  of  how  to  approximate  one  linear  system  by  another.  The  full 
generality  of  the  technique  of  [Pi]  was  unknown. 

Prior  characterizations  of  solvability,  often  amounted  to  being  able  to  solve 
the  problem.  In  [P6],  the  principal  investigator  gave  the  first  explicit, 
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verifiable,  characterization  of  solvability  in  the  cases  when  A,B, f  are  either  real 
analytic  or  3n- times  differentiable.  Necessary  conditions  are  also  given  if  A,B,f 
are  2n-times  differentiable.  These  characterizations  are  stated  directly  in  terms 
of  pointwise  rank  conditions  involving  the  arrays  in  (4).  Thus  they  may  be 
verified  without  performing  any  coordinate  changes. 

One  consequence  of  [P6]  is  a  proof  that  the  numerical  method  of  [PI]  works  for 
all  smooth  linear  time  varying  implicit  solvable  systems  (3)  ard  [P4]  can  always  be 
used  to  find  the  initial  conditions. 

Another  consequence  of  [P6]  is  a  characterization  of  when  the  solution 
manifolds  of  two  linear  time  varying  singular  systems  are  close.  This  is  the  first 
general  approximation  result  to  appear  for  the  linear  time  varying  case  and  opens 
the  door  to  the  development  of  guidelines  for  model  simplification. 

The  papers  [PI].  rp4],  [P6]  finally  put  the  numerical  and  analytic  solution  of 
(3)  on  a  firm  footing.  Techniques,  for  the  first  time,  were  now  available  to  study 
the  approximation  and  simplification  of  general  linear  time  varying  implicit 
systems. 

The  method  just  discussed  pays  a  price  for  its  extra  generality.  It  requires 
mere  computational  effort  at  each  step  than,  for  example,  a  BDF  method.  However, 
many  circuit  and  control  problems  have  special  structure,  for  example  semi -explicit 
form  as  in  (1).  In  [PS] ,  the  principle  investigator  showed  how  to  exploit  this 
structure  to  achieve  an  additional  order  of  magnitude  reduction  in  computational 
effort.  Also,  in  [P9]  an  example  was  given  that  showed  that  the  algorithmic 
changes  which  lead  to  the  reduction  also  enabled  one  to  use  the  method  to  find  a 
singular  arc  in  a  control  problem  that  did  not  meet  all  the  assumptions  for  the 
theory  of  [P6] .  This  suggests  that  the  general  approach  may  have  even  greater 
applicability  than  that  shown  in  [P6].  This  problem  is  currently  under 
investigation  by  a  new  research  assistant. 
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Many  control  problems  are  nonlinear.  The  results  of  [P6]  provide  techniques 
that  can  be  utilized  on  some  of  these  problems.  In  [Pll],  the  principal 
investigator  studied  the  control  problem 

A(t)x*(t)  +  (B(t)  +  u(t)C(t))x(t)  =  D(t)v(t)  (6) 

Algorithms  were  given  to  determine  when  the  solution  x  of  (6)  is  a  continuous 
function  of  the  control  u  and  its  derivative. 

Petzold  and  Gear  [R4],  [R5]  have  shewn  that  most  higher  index  systems  (3)  can 
be  changed,  at  least  locally,  to  an  index  two  system  by  the  use  of  linear  time 
varying  coordinate  changes.  Thus  the  index  two  case  is  of  considerable  interest. 

In  [C2]  the  principal  investigator  had  given  both  sufficient  conditions  under  which 
an  index  two  linear  time  'varying  singular  system  was  solvable  and  a 
characterization  of  the  solutions.  In  [P2]  the  principal  investigator  weakened  the 
assumptions  of  [C2]  and  obtained  a  similar  result.  A  series  of  examples  in  [P2] 
showed  that  additional  weakening  is  probably  not  possible.  These  examples  shed 
light  on  the  behavior  of  systems  which  are  index  two  but  have  higher  index  in  a 
different  coordinate  system  and  have  proved  useful  in  testing  numerical  algorithms. 

There  are  two  natural  types  of  coordinate  changes  to  make  when  studying  (3) ; 
letting  x  =  Q(t)y  and  multiplication  of  (3)  on  the  left  by  P(t).  That  neither  of 
these  effect  the  analysis  of  [PI],  [P4],  [P6],  [C4]  is  one  reason  that  approach  is 
so  useful. 

It  is  known  that  the  question  of  whether  BDF's  converge  is  unaltered  by  time 
varying  P  and  constant  Q  but  can  be  altered  by  time  varying  Q.  The  research 
assistant  K.  Clark  has  defined  a  modified  backwards  differentiation  formula  (MBDF) 
in  [P5].  This  method  bas  several  interesting  properties.  For  linear  time 


invariant  systems  and  those  in  the  standard  canonical  form  [C6]  it  is  the  same  as 
the  BDF.  However,  the  convergence  properties  of  the  MBDF  are  unchanged  by  constant 
P  and  time  varying  Q,  but  can  be  altered  by  time  varying  P.  Thus  the  MBDF  methods 
provide  a  nice  symmetry  to  the  theory.  The  methods  of  [PI],  [P4],  [C4]  are 
unaffected  by  time  varying  P  and  Q.  Additional  results  on  when  the  BDF  and  MBDF 
methods  work  is  given  in  [P7],  [P8],  [P12].  This  unifies  and  extends  seme  of  the 
work  of  [Rl] . 

A  somewhat  surprising  development  was  the  construction  of  a  singular  system 
for  which  BDF  converged  but  after  differentiation  of  the  constraints,  BDF  did  not 
converge  [P8] .  Since  differentiation  of  constraints  is  often  used  in  practice, 
this  has  implications  for  the  solution  of  systems  with  index  greater  than  two. 

From  this  research  has  emerged  a  much  better  understanding  of  why  BDF  methods 
work  on  seme  problems  and  not  others.  In  [P12]  is  the  beginning  of  the  first 
truely  general  theory  of  convergence  of  BDF. 

The  increasing  interest  in  implicit  systems  within  the  mathematical  and 
engineering  communities  is  illustrated  by  the  recent  special  sessions  held  on  this 
topic  and  invitations  to  the  principal  investigator.  Several  of  these  are 
described  in  the  travel  section.  In  addition,  the  paper  [P2]  will  appear  in  a 
special  issue  of  Circuits  Systems  §  Signal  Processing  on  semistate  equations. 

The  research  of  the  principal  investigator  and  others  on  singular  systems  is 
beginning  to  have  the  desired  effect  of  making  this  approach  accessible  and  useful 
to  scientists  and  engineers.  Examples  are  the  recent  papers  on  robotics  [R7] , 
chemical  kinetics  [R8],  moving  grids  in  partial  differential  equations  [R8],  [RIO], 
flight  control  [Rl],  [R2],  mechanics  [Rll],  [R12],  and  other  applications  involving 
the  method  of  lines  [R8],  [RIO],  [ Rl 2 ] . 


The  research  reported  here  has  also  laid  the  groundwork  for  consideration  of 
new  approaches  to  attacking  important  nonlinear  problems.  These  approaches  are  now 
being  examined. 

TRAVEL 

Travel  funds  from  this  grant  were  used  to  enable  the  principal  investigator  to 
attend  the  Society  for  Industrial  and  Applied  Mathematics  (SLAM)  1984  National 
Summer  Meeting  in  Seattle,  Washington,  July  16-20.  A  paper  concerning  this 
research  was  presented.  The  meeting  provided  an  opportunity  to  discuss  the 
proposed  research  with  experts  from  numerical  analysis,  control  theory,  and 
differential  equations. 

July  30  -  August  3,  1984;  the  principal  investigator  attended  and  gave  an 
invited  paper  [Pi]  at  the  American  Mathematical  Society  (AMS)  -  SIAM  Research 
Conference  on  "Linear  Algebra  and  Its  Role  in  Systems  Theory".  Researchers  from 
Engineering  and  Mathematics  with  an  interest  in  control  theory,  systems  theory, 
linear  algebra,  and  numerical  linear  algebra  were  present.  Recent  results  by  the 
principal  investigator  on  the  linear  algebra  problems  that  arise  in  the  numerical 
solution  of  the  implicit  systems  (3)  were  presented. 

December  12-14,  1984;  the  principal  investigator  attended  and  gave  an  invited 
paper  [P3]  at  the  23™  IEEE  Conference  on  Decision  and  Control  as  part  of  a  special 
session  on  generalized  state  space  systems. 

June  9-15,  1985;  the  principal  investigator  attended  and  gave  an  invited  talk 
[P4 ]  at  the  1985  Mathematical  Theory  of  Networks  and  Systems  Conference  in 
Stockholm ,  Sweden . 

June  18-23,  1985;  the  principal  investigator  visited  the  University  of 
Bologna,  Italy.  He  gave  a  seminar  on  recent  results  cn  singular  systems  and  had 
.mathematical  discussions  with  Professor  Angelo  Favini  about  operator  theoretic 
questions  involving  implicit  systems  of  differential  equations;  While  considering 


these  questions,  the  principal  investigator  was  able  to  prove,  by  operator 
theoretic  means,  one  of  the  key  lemmas  in  [P6J.  This  trip  was  partly  supported  by 
AFOSR  and  partly  by  the  Italian  government. 

June  24-26,  1985;  the  principal  investigator  attended  the  1985  SIAM  National 
meeting  in  Pittsburgh,  PA.  He  gave  an  invited  talk  in  the  special  session  on 
differential -algebraic  equations. 

January  7-11,  1986;  the  principal  investigator  attended  the  1986  National 
American  Mathematical  Society  Meeting  in  New  Orleans.  He  gave  an  invited  talk  in 
[P10]  in  the  special  session  on  Operator  Methods  in  Control. 

July  27-August  2,  1986;  the  principal  investigator  attended  and  presented  a 
paper  at  the  1986  Ordinary  Differential  Equations  Conference  in  Albuquerque.  This 
was  a  large  meeting  devoted  to  the  numerical  solution  of  differential  equations. 
There  were  several  people  attending  from  universities,  national  laboratories,  and 
corporations  that  were  interested  in  singular  systems. 

August  12-14,  1986;  the  principal  investiagor  attended  and  presented  a  paper 
at  the  SIAM  Meeting  on  Linear  Algebra  in  Signals,  Systems,  and  Control  in  Boston, 
Mass.  He  was  also  on  the  organizing  committee  for  this  meeting.  Approximately  400 
mathematicians  and  engineers  attended. 

In  addition  to  AFOSR  sponsored  travel ,  the  principal  investigator  made  two 
trips  to  Lawrence  Livermore  National  Laboratory  ( LLNL) ,  Livermore,  California  where 
he  is  a  Participating  Guest  with  a  DOE  Q  clearance.  The  trips  were  July  19-20, 

1986,  and  December  11-12,  1986.  The  principal  investigator  consulted  with 
Dr.  Linda  Petzold  of  the  Mathematical  Computation  Group,  Dr.  K.  Brenan  who  was 
visiting  from  Aerospace  Corporation,  and  B.  Leimkuhler  from  the  University  of 
Illinois.  The  discussions  were  on  the  numerical  solution  of  differential  algebraic 
systems  that  arise  in  various  applications.  These  trips  were  paid  for  by  LLNL. 


OTHER  PROJECT  PERSONNEL 
Research  Assistant: 

The  Research  Assistant,  Mr.  Kenneth  Clark,  worked  on  a  Ph.D.  in  Applied 
Mathematics  under  the  direction  of  the  principal  investigator.  Mr.  Clark  developed 
some  nice  results  on  the  relationship  between  the  convergence  of  backward 
differentiation  formulas  and  the  types  of  transformations  needed  to  put  an  implicit 
system  into  a  canonical  form.  These  results  are  reported  in  [P5],  [P7],  [P8], 

[P12] .  He  graduated  in  June,  1986.  An  abstract  of  his  thesis  follows. 

Abstract  of  Research  Ph.D.  Thesis 

TITLE:  Difference  Methods  for  the  Numerical  Solution  of  Nbnautononous  Singular 
Linear  Systems. 

ABSTRACT.  Several  aspects  of  the  numerical  and  analytical  treatment  of  singular 
linear  systems 

A(t)x'  +  B(t)x  =  f(t) 


are  considered.  Here  A(t)  is  assumed  to  be  identically  singular,  and  the 
differential  equation  is  considered  on  the  closed  interval  [0,T].  First,  a  class 
of  modified  difference  schemes  is  derived  for  the  numerical  solution  of  the  system. 
These  methods  are  analyzed  on  several  classes  of  problems  which  frequently  arise, 
and  in  particular  their  convergence  and  stability  is  proved  for  the  classes 
considered.  The  classes  we  examine  include  those  transformable  to  constant 
coefficients,  implicitly  defined  index  one  systems,  and  index  two  systems  in 
semi-explicit  form.  We  show  that  all  of  the  classes  investigated  in  this  paper  can 
be  treated  in  a  mere  general  framework  involving  a  special  case  of  the  standard 


canonical  form  for  time  varying  singular  linear  systems,  and  that  the  structure  of 
the  form  is  the  key  element  in  the  analysis. 
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